Cultural Context-Aware Models and IT Applications for the Exploitation of Musical Heritage

Information engineering has always expanded its scope by inspiring innovation in different scientific disciplines. In particular, in the last sixty years, music and engineering have forged a strong connection in the discipline known as “Sound and Music Computing”. Musical heritage is a paradigmatic case that includes several multi-faceted cultural artefacts and traditions. Several issues arise from the analog-digital transfer of cultural objects, concerning their creation, preservation, access, analysis and experiencing. The keystone is the relationship of these digitized cultural objects with their carrier and cultural context. The terms “cultural context” and “cultural context awareness” are delineated, alongside the concepts of contextual information and metadata. Since they maintain the integrity of the object, its meaning and cultural context, their role is critical. This thesis explores three main case studies concerning historical audio recordings and ancient musical instruments, aiming to delineate models to preserve, analyze, access and experience the digital versions of these three prominent examples of musical heritage. The first case study concerns analog magnetic tapes, and, in particular, tape music, a particular experimental music born in the second half of the XX century. This case study has relevant implications from the musicology, philology and archivists’ points of view, since the carrier has a paramount role and the tight connection with its content can easily break during the digitization process or the access phase. With the aim to help musicologists and audio technicians in their work, several tools based on Artificial Intelligence are evaluated in tasks such as the discontinuity detection and equalization recognition. By considering the peculiarities of tape music, the philological problem of stemmatics in digitized audio documents is tackled: an algorithm based on phylogenetic techniques is proposed and assessed, confirming the suitability of these techniques for this task. Then, a methodology for a historically faithful access to digitized tape music recordings is introduced, by considering contextual information and its relationship with the carrier and the replay device. Based on this methodology, an Android app which virtualizes a tape recorder is presented, together with its assessment. Furthermore, two web applications are proposed to faithfully experience digitized 78 rpm discs and magnetic tape recordings, respectively. Finally, a prototype of web application for musicological analysis is presented. This aims to concentrate relevant part of the knowledge acquired in this work into a single interface. The second case study is a corpus of Arab-Andalusian music, suitable for computational research, which opens new opportunities to musicological studies by applying data-driven analysis. The description of the corpus is based on the five criteria formalized in the CompMusic project of the University Pompeu Fabra of Barcelona: purpose, coverage, completeness, quality and re-usability. Four Jupyter notebooks were developed with the aim to provide a useful tool for computational musicologists for analyzing and using data and metadata of such corpus. The third case study concerns an exceptional historical musical instrument: an ancient Pan flute exhibited at the Museum of Archaeological Sciences and Art of the University of Padova. The final objective was the creation of a multimedia installation to valorize this precious artifact and to allow visitors to interact with the archaeological find and to learn its history. The case study provided the opportunity to study a methodology suitable for the valorization of this ancient musical instrument, but also extendible to other artifacts or museum collections. Both the methodology and the resulting multimedia installation are presented, followed by the assessment carried out by a multidisciplinary group of experts

Not just paper: enhancement of archive cultural heritage

Oral archives and digital technologies have gone hand-in-hand for a very long time. Both sides benefit from this interdisciplinary junction: technology enhances the preservation and diffusion of oral materials, while exploiting them to develop cutting-edge tools for their treatment. This chapter deals with an Italian instantiation of this mutual relationship: the Archivio Vi.Vo. project. Offering innovative solutions concerning metadata, audio restoration, description , and access, Archivio Vi.Vo. aims to build an online platform to host the oral archives from Tuscany. The project is powered by CLARIN-IT, which guarantees its compliance with standards and offers resources for data access and discov-erability. Archivio Vi.Vo. has not been built from scratch: it is instead a cross-fertilization of previous initiatives and research projects (e.g., the Gra.fo project). Moreover, the chapter presents the related, contemporary work of a multidisciplinary group striving to synthesize a Vademecum for future generations of oral archive researchers. Lastly, a brief list of tentative ideas for future developments of the Archivio Vi.Vo. platform will be presented

Cultural Context-Aware Models and IT Applications for the Exploitation of Musical Heritage

Information engineering has always expanded its scope by inspiring innovation in different scientific disciplines. In particular, in the last sixty years, music and engineering have forged a strong connection in the discipline known as “Sound and Music Computing”. Musical heritage is a paradigmatic case that includes several multi-faceted cultural artefacts and traditions. Several issues arise from the analog-digital transfer of cultural objects, concerning their creation, preservation, access, analysis and experiencing. The keystone is the relationship of these digitized cultural objects with their carrier and cultural context. The terms “cultural context” and “cultural context awareness” are delineated, alongside the concepts of contextual information and metadata. Since they maintain the integrity of the object, its meaning and cultural context, their role is critical. This thesis explores three main case studies concerning historical audio recordings and ancient musical instruments, aiming to delineate models to preserve, analyze, access and experience the digital versions of these three prominent examples of musical heritage. The first case study concerns analog magnetic tapes, and, in particular, tape music, a particular experimental music born in the second half of the XX century. This case study has relevant implications from the musicology, philology and archivists’ points of view, since the carrier has a paramount role and the tight connection with its content can easily break during the digitization process or the access phase. With the aim to help musicologists and audio technicians in their work, several tools based on Artificial Intelligence are evaluated in tasks such as the discontinuity detection and equalization recognition. By considering the peculiarities of tape music, the philological problem of stemmatics in digitized audio documents is tackled: an algorithm based on phylogenetic techniques is proposed and assessed, confirming the suitability of these techniques for this task. Then, a methodology for a historically faithful access to digitized tape music recordings is introduced, by considering contextual information and its relationship with the carrier and the replay device. Based on this methodology, an Android app which virtualizes a tape recorder is presented, together with its assessment. Furthermore, two web applications are proposed to faithfully experience digitized 78 rpm discs and magnetic tape recordings, respectively. Finally, a prototype of web application for musicological analysis is presented. This aims to concentrate relevant part of the knowledge acquired in this work into a single interface. The second case study is a corpus of Arab-Andalusian music, suitable for computational research, which opens new opportunities to musicological studies by applying data-driven analysis. The description of the corpus is based on the five criteria formalized in the CompMusic project of the University Pompeu Fabra of Barcelona: purpose, coverage, completeness, quality and re-usability. Four Jupyter notebooks were developed with the aim to provide a useful tool for computational musicologists for analyzing and using data and metadata of such corpus. The third case study concerns an exceptional historical musical instrument: an ancient Pan flute exhibited at the Museum of Archaeological Sciences and Art of the University of Padova. The final objective was the creation of a multimedia installation to valorize this precious artifact and to allow visitors to interact with the archaeological find and to learn its history. The case study provided the opportunity to study a methodology suitable for the valorization of this ancient musical instrument, but also extendible to other artifacts or museum collections. Both the methodology and the resulting multimedia installation are presented, followed by the assessment carried out by a multidisciplinary group of experts.L’ingegneria dell’informazione ha sempre abilitato innovazione in moltissimi settori scientifici. In particolare, negli ultimi sessant’anni, la ricerca nel campo della Sound and Music Computing, ha coniugato musica e ingegneria in un’unica disciplina. Il patrimonio musicale è un caso paradigmatico che include numerose tipologie di reperti e di tradizioni. Il trasferimento nel dominio digitale di oggetti culturali implica diversi problemi legati alla loro creazione, conservazione, accesso, analisi e fruizione. Il cardine è la relazione degli oggetti culturali digitalizzati con il loro supporto originario e con il loro contesto culturale. In questa tesi, i concetti di cultural context e cultural context awareness sono descritti in relazione ai metadati e alle informazioni contestuali necessarie per mantenere l’integrità, il significato e il contesto culturale dell'oggetto digitalizzato. Questo lavoro approfondisce tre studi di caso riguardanti i documenti sonori storici e gli strumenti musicali antichi, con l’obiettivo di formalizzare dei modelli per la conservazione, l’analisi, l’accesso e la fruizione del patrimonio musicale. Il primo studio di caso riguarda le registrazioni su nastro magnetico di tape music, una particolare musica sperimentale della seconda metà del Novecento, che ha rilevanti implicazioni dal punto di vista musicologico, filologico e archivistico. In questo caso, il supporto ha un ruolo di straordinaria importanza e una stretta relazione con il suo contenuto, che può facilmente perdersi durante la digitalizzazione o la fruizione. È stato sviluppato un insieme di strumenti automatici basati su tecniche di intelligenza artificiale per supportare musicologi e tecnici audio nel loro lavoro. È quindi stato studiato il problema filologico relativo alla creazione dello stemma codicum nei documenti sonori digitalizzati, ben considerando le peculiarità della tape music. A tale scopo, è stato sviluppato un algoritmo basato su tecniche di filogenetica, successivamente testato con ottimi risultati. A seguire, la tesi definisce una innovativa metodologia per una fruizione storicamente fedele dei documenti sonori digitalizzati, in grado di valorizzare le informazioni contestuali, il supporto originale e il suo dispositivo di riproduzione. Sulla base di questa, è stata sviluppata un’app Android che virtualizza un magnetofono, presentata assieme alla valutazione di un gruppo multidisciplinare di esperti. Inoltre, sono presentate due applicazioni che propongono la virtualizzazione rispettivamente di un grammofono per dischi a 78 giri e un magnetofono. La tesi propone anche una interfaccia web per l’analisi musicologica, che cerca di racchiudere in un’unica applicazione gli strumenti e le conoscenze acquisite. Un secondo studio di caso è basato su di un corpus di musica Araba-Andalusa, creato appositamente per soddisfare le esigenze della computational musicology, e finalizzato a uno studio data-driven di questa importante tradizione musicale. Il corpus è presentato sulla base di cinque criteri formalizzati nel progetto CompMusic dell’Università Pompeu Fabra di Barcellona: scopo, copertura, completezza, qualità e riusabilità. Sono stati sviluppati quattro notebook Jupyter al fine di fornire uno strumento per l’utilizzo e l’analisi del corpus. Il terzo studio di caso concerne un eccezionale strumento musicale antico: un flauto di Pan esposto al Museo di Scienze Archeologiche e d’Arte dell’Università di Padova. È stata progettata e realizzata un’installazione multimediale per la valorizzazione di questo straordinario reperto, in grado di fornire ai visitatori del museo la possibilità di interagire con la sua copia digitale e la sua storia. Questo studio ha permesso di approfondire e formalizzare una metodologia per la valorizzazione di questo strumento musicale antico, che può essere estesa ad altri reperti o collezioni museali. Sia la metodologia, sia l’installazione sono state valutate da un gruppo multidisciplinare di esperti

Un modello per la fruizione filologica di documenti sonori storici: virtualizzazione web-based di un grammofono per la riproduzione di dischi fonografici digitalizzati

La tesi descrive un modello per la fruizione filologica di dischi fonografici e l’implementazione della relativa applicazione web. Vengono analizzati i supporti in gommalacca e descritti gli errori più comuni nelle operazioni di digitalizzazione. Il secondo capitolo illustra l’ideazione e lo sviluppo del grammofono, prendendo in esame la tecnologia utilizzata; viene quindi presentata l’effettiva implementazione dell’applicazione. La conclusione propone possibili sviluppi futur

Building a Home for Italian Audio Archives

Audio and audiovisual archives are at the crossroads of different fields of knowledge, yet theyrequire common solutions for both their long-term preservation and their description, availability,use and reuse. Archivio Vi.Vo. is an Italian project financed by the Tuscany Region, aiming to(i) explore methods for long-term preservation and secure access to oral sources and (ii) developan infrastructure under the CLARIN-IT umbrella offering several services for scholars from dif-ferent domains interested in oral sources. This paper describes the project’s infrastructure and itsmethodology through a case study on the Caterina Bueno’s audio archive

Tape music archives: From preservation to access

Journal article

Nawba recognition for Arab-Andalusian music using templates from music scores

Comunicació presentada a: the 15th Sound and Music Computing Conference (SMC2018). Sonic crossing, celebrat a Limassol, Xipre, del 4 al 7 de juliol de 2018.The Arab-Andalusian music is performed through nawab¯at (plural of nawba), suites of instrumental and vocal pieces ordered according to their metrical pattern in a sequence of increasing tempo. This study presents for the first time in literature a system for automatic recognition of nawba for audio recordings of the Moroccan tradition of Arab- Andalusian music. The proposed approach relies on template matching applied to pitch distributions computed from audio recordings. The templates have been created using a data-driven approach, utilizing a score collection categorized into nawab¯at. This methodology has been tested on a dataset of 58 hours of music: a set of 77 recordings in eleven nawab¯at from the Arab-Andalusian corpus collected within the CompMusic project and stored in Dunya platform. An accuracy of 75% on the nawba recognition task is reported using Euclidean distance (L2) as distance metric in the template matching

Community-based survey and oral archive infrastructure in the Archivio Vi.Vo. project

Audio and audiovisual archives are at the crossroads of different fields of knowledge, yet theyrequire common solutions for both their long-term preservation and their description, availability,use and reuse.Archivio Vi.Vo.is an Italian project financed by the Region of Tuscany, aimingto: (i) explore methods for long-term preservation and secure access to oral sources, and (ii)develop an infrastructure under the CLARIN-IT umbrella offering several services for scholarsfrom different domains interested in oral sources. This paper describes the project’s infrastructureand its methodology through a case study on Caterina Bueno’s audio archive